Adaptive background image updating

ABSTRACT

A method compares a background image to input images to determine a similarity scores λ for each input image. Then, the background image is updated only if the similarity score for a particular image is less than a predetermined threshold. Presumably, any pixel whose color does not change is part of a static background, and any pixel that does change is part of a moving object. The similarity score controls when input images are scored and the manner the background image is updated.

FIELD OF THE INVENTION

This invention relates generally to image processing, and moreparticularly to updating a background image representing a changingbackground in a scene.

BACKGROUND OF THE INVENTION

In vision systems, it is a fundamental and crucial problem to extractmoving objects from a video. Typical applications in which objectextraction is necessary include surveillance, traffic monitoring,teleconferencing, human-machine interface, and video editing. There aremany other image processing applications where it is important to beable to distinguish moving pixels from static pixels. It is common torefer to the moving object as a ‘foreground’ object. However, it shouldbe understood that the relative depth at which the object moves withrespect to the ‘background’ is not a factor.

Background subtraction is the most common technique for moving objectextraction. The idea is to subtract a current image from a static imagerepresenting the ‘background’ in a scene. The subtraction should revealthe silhouettes of moving objects. Background subtraction is performedtypically during a pre-processing step to object recognition andtracking. Most prior art background subtraction methods are based ondetermining a difference in pixel intensity values between two images.

Although prior art methods work well, they are susceptible to global andlocal color in intensity. If the color in portions of the background ofan image changes, then the pixel intensity levels change accordingly,and those portions may be misclassified being associated with theforeground or a moving object. These changes cause the subsequentprocesses, e.g., recognition and tracking, to fail because accuracy andefficiency are crucial to those tasks.

In the prior art, computational barriers have limited the complexity ofreal-time video processing applications. As a consequence, most priorart systems were either too slow to be practical, or restricted tocontrolled situations.

Recently, faster computers have enabled more complex and robust systemsfor analyzing videos in real-time. Such systems can reveal real worldprocesses under varying conditions. A practical system should not dependon the placement of cameras, the content of the scene, or lightingeffects. The system should be able to track slow moving objects in acluttered scene with a moving background, overlapping or occludedobjects, objects in shadows and subject to lighting changes, as oldobject leave the scene, and new objects enter.

Non-adaptive background subtraction methods are undesirable because theyrequire manual initialization to acquire the static background image.Without initialization, errors in the background accumulate over time,making non-adaptive methods useless for unsupervised, long-term trackingapplications where there are significant changes in the background ofthe scene.

Adaptive background subtraction averages pixel intensities over time togenerate the background image, which is an approximation of thebackground at any one time. While adaptive background subtraction iseffective in situations where objects move continuously and thebackground is visible a significant portion of the time, it fails whenthere are many moving objects, particularly if the moving objects moveslowly. Adaptive methods also fail to handle multi-modal, or multiplealternating backgrounds, recover slowly when the background is uncoveredand a single, predetermined threshold is used for the entire scene.

Changes in lighting cause many background subtraction methods to fail.Ridder et al. in “Adaptive background estimation and foregrounddetection using Kalman-filtering,” ICRAM (International Conference onRecent Advances in Mechatronics), 1995, modeled each pixel with a Kalmanfilter. This modeling made their system less sensitive to lightingchanges. While that method does have a pixel-wise automatic threshold,it still recovers slowly.

Koller et al., in “Towards robust automatic traffic scene analysis inreal-time,” Proceedings of the International Conference on PatternRecognition, 1994, adapted Kalman filtering to an automatic trafficmonitoring application.

A multi-class statistical model can also be used for tracking objects byapplying a single Gaussian filter to each image pixel. After properinitialization with a static indoor scene, reasonable results can beobtained. Results for outdoor scenes are not available.

Friedman et al. in “Image segmentation in video sequences: Aprobabilistic approach,” Proc. of the Thirteenth Conf. on Uncertainty inArtificial Intelligence, 1997, described a pixel-wiseexpectation-maximization (EM) method for detecting moving vehicles.Their method explicitly distributes pixel values into three classesrepresenting the road, shadows, and vehicles. It is unclear whatbehavior their system would exhibit for pixels with otherclassifications.

Instead of using predetermined distributions, Stauffer et al., in“Adaptive background mixture models for real-time tracking,” Proc. IEEEConf. on Computer Vision and Pattern Recognition, Vol. 2, 1999, modeledeach pixel as a mixture of Gaussian filters. Based on the persistenceand the variance of each mixture, they determined the Gaussian filtersthat could correspond to background distributions. Pixels that do notfit the background distributions are considered foreground unless thereis a Gaussian filter with sufficient and consistent evidence to supporta background classification. Their updating has four problems. Itconsiders every pixel in the image. The pixels are processedindependently. It uses a constant learning coefficient, and thebackground image is always updated for every input image, whether or notupdating is actually required. This can be quite time consuming, if theupdating is at the frame rate.

Bowden et al., in “An improved adaptive background mixture model forreal-time tracking with shadow detection,” Proc. 2nd European Workshopon Advanced Video Based Surveillance Systems, AVBS01, September 2001,described a model based on the method of Grimson et al. They used an EMprocess to fit a Gaussian mixture model for the update equations duringinitialization.

Horprasert et al., in “A statistical approach for real-time robustbackground subtraction and shadow detection,” IEEE ICCV'99 FRAME-RATEWORKSHOP, 1999, described a computational color model that considersbrightness distortion and chromaticity distortion to distinguish shadingbackground from the ordinary background or moving foreground objects.They also described a process for pixel classification and thresholdselection.

In general, there are three major problems with prior art backgroundimage updating. First, every input image causes a background update.This wastes time. Second, every pixel in each image is considered duringthe updating. This wastes even more time. Third, the manner in which thebackground image is updated is fixed. This is sub-optimal. The inventionaddresses all three problems.

SUMMARY OF THE INVENTION

The invention provides a method that compares a background image toinput images to determine a similarity scores λ for each input image.Then, the background image is updated only if the similarity score for aparticular image is less than a predetermined threshold. Presumably, anypixel whose color does not change is part of a static background, andany pixel that does change is part of a moving object. The similarityscore controls when input images are scored and the manner thebackground image is updated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a method for adaptively updating abackground image representing a changing background in a scene accordingto the invention;

FIGS. 2A-C are block diagrams of a background image, an input image, andan object mask, respectively;

FIG. 3 is a graph of color evaluation in an RGB color domain accordingto the invention;

FIG. 4 is a block diagram of a process for estimating parameters of acolor model according to the invention;

FIG. 5A is a block diagram of an update controller;

FIG. 5B is a timing diagram of continuous updates according to theinvention;

FIG. 5C is a timing diagram of impulse updates according to theinvention;

FIG. 6 is a block diagram of an update controller using interlacedvideo;

FIG. 7 is a block diagram of an update controller with identicalregions; and

FIG. 8 is a block diagram of an update controller with random sets ofpixels.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Introduction

In adaptive background image updating according to the invention, abackground image is compared to input images to determine a similarityscores λ for each input image. Then, the background image is updatedonly if the similarity score for a particular image is less than apredetermined threshold. Presumably, any pixel whose color does notchange is part of a static background, and any pixel that does change ispart of a moving object. The similarity score controls when input imagesare scored and the manner the background image is updated.

Thus, the first problem is to determine the background image. Duringlong-term tracking, the background image should be updated according tochanges in lighting condition of the scene. Thus, the second problem ishow much to update the background image according to similarity score.

It should be noted that the differences in images can be due to a movingobject entering the scene, and then stopping. In such a case, it isnecessary to make the stopped object part of the background. In such ascase, the color values of the stopped object can simply replace thecolor values of what was the underlying background. For much subtlechanges in color, due to the direction and intensity of light, e.g., themoving sun or absence thereof, some type of blending is preferred. Thethird and fourth problems are to determine a rate of scoring andpotential updating, and the minimum number of necessary pixels thatshould be updated.

Adaptive Updating Method

FIG. 1 shows a method for adaptively updating a background image B 108representing a background in a scene according to the invention. Theinput to the method are images I 101 acquired of the scene, e.g., avideo, see FIG. 2A.

A background generation unit initializes 110 the background image 108 asdescribed in greater detail below, see FIG. 2B. We distinguish ourbackground image 108 from a prior art background image as follows. Asused in the prior art, a background image is typically a one-time,static image of a background scene. While in an adaptively updatedbackground image according to the invention, only portions of thebackground that experience color changes are updated 170 only whenneeded, over time. The background image 108 can be stored in a memory105 of a computer system while the computer implemented method asdescribed herein operates on the background image.

Background Image Initialization

The initial background image 108 can be constructed from the first, orfirst few frames of the video 101. For example, the background image isthe first image, or an average of the first few images. Alternatively,the background image is initialized as a blank or random image. In thiscase, the updating as described herein, will automatically replace theinitial background image when the first input image is processed.

Reference Model

In the preferred embodiment, the background image is 108 is derived froman intermediate reference model (RM) 109 also stored in the memory 105.It should be understood that the reference model is not required for thebasic operation of the invention.

The reference model 109 is in the form of pixel-wise mixture ofprobability distribution functions to support multi-model backgrounds.We model the history of a color channel by expressing each pixel by amixture of Gaussian distributions.

The color value of a pixel in the current input image I 101 is denotedby I(p). The probability of observing a current pixel color value for asingle channel at frame t isP(I(p),t)=Σ_(n) ^(N) w _(n)(t)g(I(p), μ_(n)(t), σ_(n) ²(t))

where N is the number of distributions, w_(n)(t) is the weight of then^(th) Gaussian distribution in the mixture, μ_(n)(t) and σ^(n) ₂(t) arethe mean value and the variance of the n^(th) Gaussian distribution inthe mixture at frame t respectively, and g is a Gaussian probabilitydensity function

${g\left( {{I(p)},\mu,\sigma^{2}} \right)} = {\frac{1}{\sqrt{2{\pi\sigma}}}{{\mathbb{e}}^{- \frac{{\lbrack{{I{(p)}} - \mu}\rbrack}^{2}}{\sigma^{2}}}.}}$

In the above formulation, the color channels are assumed to beindependent from each other. A covariance matrix could also be used.However, the matrix provides only a minimal improvement with aconsiderable increase in the computational complexity.

Object Tracker

The object tracker 120 compares each input images I 101 to thebackground image B 108. The purpose of the object tracker is to identifypixels that are associated with moving objects. To reduce thecomputational load, these pixels should not be considered during theupdating 170. The object tracker generates a binary mask M 202, see FIG.2C. The object tracker 120 can use a well known means-shift operation onthe previous and current images to track pixels associated with movingobjects. The purpose of the object tracker is not to consider pixelsthat are part of moving objects for updating purposes.

A distance d(p) between a pixel in the current input image 101 and thecorresponding pixel in the background image 108 is defined asd(p)=|B(p)−I(p)| ω _(n)*(t,p),

where w_(n)*(t,p) stands for a weight of the best distributioncorresponding to pixel p. If this distance is larger than a distancethreshold, then the pixel is marked with a binary 1 as a possibleforeground pixel, otherwise it is marked as a zero. It should be notedthat it may be advantageous to use the previous image instead of thecurrent to determine the mask 202. In addition, it should be understoodthat the object tracker is not required for the basic operation of theinvention. For example, other sensing means can be used to determinewhether or not there is a moving object in the scene.

Sampling

The sampler 130 also takes input from the input image 101 and thebackground image 108. The sampler 130 selects certain number (S) ofpixels to compare. The selected pixels do not correspond to movingobject in the current image. The selection can be random. Alternatively,the sampling can be uniform up to a predetermined number S of pixels.Pixels corresponding to moving object are disregarded by the samplingprocess 130. Thus, foreground or moving object pixels do not influencethe updating 170. The set of the pixels obtained after sampling processis denoted by Q_(s).

Uniform selection with sampling frequencies C_(1s), C_(2s) generates theset Q asQ→{B(x_(s),y_(s),c,t),I(x_(s),y_(s),c,t)|(x_(s),y_(s))=(c_(1s)m,c_(2s)n)

M(x_(s),y_(s)t)≠1},

where B(x,y,c,t) is the background image 108, I(x,y,c,t) is the currentimage 101, M(x,y,t) is the object mask 202 obtained from the objecttracker 120 such that M it is equal to 1 if pixel (x,y,t) corresponds toan object, c is the color channel, t is the frame number, and the numberof selected pixels in the set Q is s=1, . . . , S. The output sampling130, the set Q, is used to determine 140 a similarity score λ. Thesampler 130 decreases the computational load for determining 140 thesimilarity scores. It should be noted that the sampler 130 is notrequired for the basic operation of the invention.

Determining Similarity Scores

The similarity score λ controls when input images are scored and themanner in which the background image 108 is updated. As stated above,the update only occurs if the similarity score is less than apredetermined threshold. In addition, there is no need to update pixelsin the reference model that already have a very low variance. Instead ofupdating the background image and reference model at a predetermined orfixed rate, as in the prior art, we determine image similarity scores λand control the scoring 140 and updating 170 accordingly.

For this propose, we determine the similarity score λ(t) for the set ofselected pixels Q that do not correspond to a moving object. Thesepixels were selected by the sampler 130 as described above. Thesimilarity scores are computed from color attributes of the pixels.

In order to understand the scoring process, a brief description ofattributes of color as perceived by living creatures is provided. Anumber of color models are known. One human oriented model uses hue,chrominance, and intensity. Hue is the dominant wavelength of the color,chrominance or saturation is the degree of departure of a color from aneutral color, such as white, and intensity is related to the emittedenergy of a color.

The red-green-blue (RGB) model is additive according to the intensity ofthe three basic colors. The YIQ model is a differential version of theRGB model. These two models are typically used in display devices. Thecyan-magenta-yellow-black (CMY(K)) model is subtractive and is usedfrequently for printing.

The YUV model, Y represents intensity, and UV represents hue andsaturation. Here UV is also called chrominance. The HIS model is the YUVmodel in polar coordinates that expresses hue as an angle, saturation asa radial component, and intensity veridical to the UV plane. Other colormodels, such as the HIS, HSV, CIE, LAB, and XYZ models can also be used.The scoring according to the invention can consider any of these colorattributes, or combinations thereof.

For example as shown in FIG. 3 for the RGB model, the determining 140evaluates the color values (R,G,B) of the selected pixels in the set Q,and expresses the similarity score as a ratio

${{\lambda(t)} = {\sum\limits_{s}^{S}{\frac{{{{{B\left( {x_{s},y_{s},c,t} \right)}{I\left( {x_{s},y_{s},c,t} \right)}}}\cos\;\theta} - {{B\left( {x_{s},y_{s},c,t} \right)}}^{2}}{{B\left( {x_{s},y_{s},c,t} \right)}}}}},$

where θ 301 is the angle between the pixel S color vector I(.) 302 andthe background color vector B(.) 303.

Alternatively, we can convert to color vectors in the YUV color domain,and evaluate similarities in Y-channel versus U and V channels as

${\lambda(t)} \equiv {\frac{\sum\limits_{s}^{S}{{{B\left( {x_{s},y_{s},Y,t} \right)} - {I\left( {x_{s},y_{s},Y,t} \right)}}}}{\begin{matrix}{{\sum\limits_{s}^{S}{{{B\left( {x_{s},y_{s},U,t} \right)} - {I\left( {x_{s},y_{s},U,t} \right)}}}} +} \\{\sum\limits_{s}^{S}{{{B\left( {x_{s},y_{s},V,t} \right)} - {I\left( {x_{s},y_{s},V,t} \right)}}}}\end{matrix}}.}$

In another method, we determine the similarity score for selected pixelsby

${\lambda(t)} \equiv {\sum\limits_{c}{\sum\limits_{s}^{S}{{{{B\left( {x_{s},y_{s},c,t} \right)} - {I\left( {x_{s},y_{s},c,t} \right)}}}.}}}$

Using machine learning as shown in FIG. 4, we can also determine colorchange parameters by training a neural network according to

I^(*)(x_(s), y_(s), c, t) = f(α_(c), I(x_(s), y_(s), c, t)), and${\lambda(t)} \equiv {\frac{1}{S}{\sum\limits_{s}{{{{B\left( {x_{s},y_{s},c,t} \right)} - {f\left( {\alpha_{c},{I\left( {x_{s},y_{s},c,t} \right)}} \right)}}}.}}}$

Using similarity training data 401, dissimilarity training data 402,object motion training data 403, and an color model 420, a neuralnetwork 410 is trained, and estimated color model parameters α_(c) 431are obtained 430 and compared with selected pixels 450 to determine thesimilarity score λ 440.

Update Controller

The purpose of the update controller is to provide the parameters 160for the update module 170, namely a learning coefficient or blendingweight α, the set Q, and an update mode M. The update controller alsodetermines when input images should be scored to lead to a potentialupdate. If the illumination in the scene is changing rapidly, then thescoring rate should approach the frame rate. Conversely, when the sceneis relatively stable, scoring can take place infrequently.

One way to interpret the similarity score λ is as follows. If anormalized score is in the range [0,1], then a score of 1 means theinput and background images are identical, and a score of zero meansthat the images are completely different. The threshold can be setanywhere in the range [0,1] to match a desired sensitivity.

We consider three rates, a frame rate r_(f) at which images areacquired, a scoring rate r_(s), which determines when the similarity ofinput images are determined, and an update rate r_(u), which reflectsthe frequency at which the background image is actually updated.According to the invention, the relationship between these rates can beexpressed as r_(u)≦r_(f)≦r_(s), whereas the relationship for the priorart is usually

$r_{u} = {\frac{r_{f}}{k}.}$If the frame rate is relatively high and the background remains stable,and k=1, then the invention provides a substantial saving in time overprior art methods because r_(u) will approach zero.

FIG. 5A show the process of the update controller 150. First determine140 the similarity score λ as described above. We presume that scoringincludes the object tracking 120 and sampling 130 steps. In practice,the scoring rate is expressed as a time interval Δt_(m) such that

$\Delta\; t_{m}\left\{ \begin{matrix}{\Delta\; t_{\max}} & {{\lambda(t)} \geq \tau_{0}} \\{\min\left( {{{\Delta\; t_{m - 1}} + 1},{\Delta\; t_{\max}}} \right)} & {{\tau_{0} > {\lambda(t)} \geq \tau_{1}},} \\1 & {\tau_{1} > {\lambda(t)}}\end{matrix} \right.$

where m is the total number of updates after initialization, τ₁<τ₀, andΔt_(max) is a maximum number of images that are processed before thenext time an image is scored. The threshold value of τ₁ can bedetermined empirically. The threshold controls the sensitivity of theupdate process. This parameter can depend on the amount of noise in theimages.

If the value of the similarity score λ(t), at time t, is less than apredetermined threshold τ₁ in step 502, then a learning or blendingparameter α is adjusted as

$\begin{matrix}{\alpha = {0.01 + \frac{\lambda(t)}{s}}} & {{\tau_{1} > {\lambda(t)}},}\end{matrix}$

-   -   where s is the number of pixels in the set Q.

The learning coefficient or blending weight alpha (α) serves as aparameter that controls the manner of updating, i.e., how much of thecolor values of the pixels in the reference model is influenced by thecolor values of the current image 101. If the color similarity scorebetween the reference model and the current frame is relatively high,then the learning coefficient is assigned a lower value. Thus, thereference model is influenced less by the current image. High values forthe learning coefficient, e.g., α=1, cause a moving object to ‘blend’into the background when the object stops moving.

Modes of Update

As shown in FIG. 5B, the input images can be scored continuously with avariable and/or, constant rate. For example, the images are alwaysscored at least at a constant lower ‘background’ rate 510, and at othertimes at a higher rate 511. For example, if the dissimilarity isrelatively high, then scoring should occur more frequently.

As shown in FIG. 5C, the scoring can also be done impulsively, asneeded, at a variable rate 521 or a constant rate 522.

Instead of updating the entire reference model, the update process canalso be applied to specific regions of the background image. We provideseveral methods for partial updates of the background image as shown inFIGS. 6-8.

For an interlaced video 601, the even-field 611 and the odd-field 612can be updated during respective update cycles 621 and 622.

For a non-interlaced video 701, an image can be partitioned 702 into Jequal sized regions 710, and the regions 710 can update sequentially720, in turn, during J update cycles.

An alternative process, partitions 810 a non-interlaced video 801 into Jnon-overlapping sets of pixels 802, that are then updated sequentially820. Here, the sets of pixels can be obtained by a random sampling ofthe background image.

Model and Background Image Update

Initially, the variances of all pixels in the model are set to a highvalue, and their weights to a small value, because none of the pixelshave sufficient statistics to make confident prediction of the actualcurrent background. The reference model is updated by comparing thecurrent pixel with existing K Gaussian distributions.

In case the color value of the current pixel is similar to the meanvalue of a distribution, it is marked as a match. The distance thresholdis set to 2.5σ to include 95% of the color values, which form the model.If none of the K distributions K<N matches the current pixel value, anew distribution is initialized.

In case of K=N, the distribution with a highest variance, i.e., a lowestconfidence, is replaced with a distribution with the current pixelsvalue as its mean value, and a large initial variance.

The means and variances of the matched distributions are quantitativelyupdated 170 using the learning coefficient α asμ_(n)(t)=[1−α]μ_(n)(t−1)+αI(p)σ_(n) ²(t)=[1−α]σ_(n) ²(t−1)+α[μ_(n)(t)−I(p)]²,

and the weights of the existing K distributions w_(n)(t) n=1, . . . ,Kare adjusted as

${w_{n}(t)} = \left\{ \begin{matrix}{{\left( {1 - \alpha} \right){w_{n}\left( {t - 1} \right)}} + \alpha} & {{{{\mu_{n}(t)} - {I(p)}}} < {2.5{\sigma_{n}(t)}}} \\{\left( {1 - \alpha} \right){w_{n}\left( {t - 1} \right)}} & {{{{{\mu_{n}(t)} - {I(p)}}} \geq {2.5{\sigma_{n}(t)}}},}\end{matrix} \right.$

where I(p) is the current color value of a pixel. This type of updatingis sometimes also known as alpha-blending.

To determine the color value of a pixel of the background image 108 fora color channel from the corresponding pixel in the reference model 109,the Gaussian distribution with the smallest variance, i.e., the highestconfidence, and the highest weight is selected. The mean of that pixelis used as the color value of the corresponding pixel in the backgroundimage 108.

Because the background image 108 is updated only when the similarity isless than a threshold value, a major reduction in overall computation isachieved, while still taking advantages of Gaussian mixture modelapproach.

Although the invention has been described by way of examples ofpreferred embodiments, it is to be understood that various otheradaptations and modifications may be made within the spirit and scope ofthe invention. Therefore, it is the object of the appended claims tocover all such variations and modifications as come within the truespirit and scope of the invention.

1. A method for adaptively updating a background image representing abackground of a scene, comprising: selecting, uniformly, a set Q of spixels in each image, wherein the set Q is Q→{B(x_(s),y_(s),c,t),I(x_(s), y_(s),c,t)|(x_(s),y_(s))=(c _(1s),m,c_(2s)n)

M(x_(s),y_(s),t)≠1}, B(x,y,c,t) is the background image, I(x,y,c,t) is aparticular image, M(x,y,t) is an object mask, c_(1s) and c_(2s) aresampling frequencies, c is a color channel, t is a frame numberassociated with the particular image, and s is a number of pixels in theset Q; determining a similarity score for the set Q of s pixels in eachof a plurality images of the scene; and updating the background imageonly if the similarity score for a particular image is less than apredetermined threshold.
 2. The method of claim 1 wherein a rate of whenthe similarity scores are determined depends on the similarity score. 3.The method of claim 1 wherein a manner in which the background isupdated depends on the similarity score.
 4. The method of claim 1further comprising: initializing the background image to a first one ofthe plurality of images.
 5. The method of claim 1 further comprising:initializing the background image to a blank image.
 6. The method ofclaim 1 wherein the background image is derived from a reference modelin a form of pixel-wise mixture of probability distribution functions.7. The method of claim 1 further comprising: identifying pixels inimages associated with moving objects; excluding the identified pixelsfrom the determining and updating.
 8. The method of claim 7 wherein theidentified pixels are represented by a binary mask.
 9. The method ofclaim 1 wherein the pixels in the set are selected randomly.
 10. Themethod of claim 1 wherein the similarity score is expressed as a ratio${\lambda(t)} = {\sum\limits_{s}^{S}{\frac{{{{B\left( {x_{s},y_{s},c,t} \right)}}{{I\left( {x_{s},y_{s},c,t} \right)}}\cos\;\theta} - {{B\left( {x_{s},y_{s},c,t} \right)}}^{2}}{{B\left( {x_{s},y_{s},c,t} \right)}}}}$where λ(t) is the similarity score for an image at time t, B is thebackground image, I is the particular image, θ an angle between colorvectors for a pixel at a location (x,y), and c is a color channel. 11.The method of claim 1 wherein the similarity score is expressed as aratio${{\lambda(t)} \equiv \frac{\left. \Sigma \middle| {{B\left( {x,y,Y,t} \right)} - {I\left( {x,y,Y,t} \right)}} \right|}{\begin{matrix}\left. \Sigma \middle| {{B\left( {x,y,U,t} \right)} - {I\left( {x,y,U,t} \right)}} \middle| + \right. \\\left. \Sigma \middle| {{B\left( {x,y,V,t} \right)} - {I\left( {x,y,V,t} \right)}} \right|\end{matrix}}},$ where λ is the similarity score for an image at time t,B is the background image, I is the particular image, (x,y) is a pixellocation, and c is a color channel.
 12. The method of claim 1 whereinthe similarity score is expressed as a ratio${{\lambda(t)} \equiv {{\sum\limits_{c}\;{\sum\;{\text{|}{B\left( {x_{s},y_{s},c,t} \right)}}}} - {{I\left( {x_{s},y_{s},c,t} \right)}\text{|}}}},$where λ is the similarity score for an image at time t, B is thebackground image, I is the particular image, (x,y) is a pixel location,and c is a color channel.
 13. The method of claim 1 wherein thesimilarity score is determined by a trained neural network.
 14. Themethod of claim 2 wherein the rate is expressed as a time interval$\Delta\; t_{m}\left\{ {{\begin{matrix}{\Delta\; t_{\max}} \\{\min\left( {{{\Delta\; t_{m - 1}} + 1},{\Delta\; t_{\max}}} \right)} \\1\end{matrix}\begin{matrix}{{\lambda(t)} \geq \tau_{0}} \\{\tau_{0} > {\lambda(t)} \geq \tau_{1}} \\{\tau_{1} > {\lambda(t)}}\end{matrix}},} \right.$ where m is a total number of updates, τ₁ andare thresholds such that τ₁<τ₀, and Δt_(max) is a maximum number ofimages that are processed before next similarity score is determined.15. The method of claim 1 further comprising: determining a blendingparameter as $\begin{matrix}{\alpha = {0.01 + \frac{\lambda(t)}{s}}} & {{\tau_{1} > {\lambda(t)}},}\end{matrix}$ where s is the number of pixels in the set Q, λ is thesimilarity score at a time t, and τ₁ is the predetermined threshold. 16.The method of claim 15 wherein a particular pixel p in the backgroundimage is updated according toB(t)=[1−α]p(t−1)+αI(t).
 17. The method of claim 1 wherein similarityscores are determined with a variable and a constant rate.
 18. Themethod of claim 1 wherein only specific regions of the background imageare updated.